WIP: Binary distributions via GitHub Releases.#98
Conversation
| name: Create GitHub Draft Release | ||
| # Only run this job for tag pushes after the R-build job completes | ||
| # successfully (also means the pushed tag matches SITK_TARGET since | ||
| # R-build validates that at the beginning). |
There was a problem hiding this comment.
Keep it simple. Just run when there is a tag starting with be. The whole comparison is really not needed.
There was a problem hiding this comment.
The tag matching is now only performed for the draft release creation job. That step should only happen when the intention is to create a release and then the pushed tag should match the one in the DESCRIPTION file so that the installer and the SimpleITK versions are kept in sync.
95a05ca to
cc5585f
Compare
|
Hey guys, It is amazing that you guys are officially releasing binaries. I think I saw that the Python binary releases of new SimpleITK versions now will come with Elastix installed too. Can we also do the same with R binaries here? Would be a nice addition since most of R wrappers of registration software out there cannot handle large images. Best, |
|
Hi @Artur-man, This is still experimental, but we are thinking of turning SimpleElastix on by default, so that will apply to all languages. Note that the two developments, SimpleElastix and R binary distribution, are independent of each other. Currently it looks like the workflow for R binary distribution is working. Would appreciate it if you could try some of the artifacts on your side. The PR testing is still running but some of the artifacts have already been uploaded here. Let us know if you test some of them and if you encounter problems. Note that the macos-ARM are not there as those are built on CircleCI and we're a bit cautious in terms of security and setting it up to allow CircleCI to automatically upload artifacts to the GitHub release. With SimpleITK 3.0 we hope that the required build resources on GitHub will be sufficient and we'll add the ARM builds here. Otherwise, we'll need to think of another solution. |
|
Thanks so much @zivy, will do. Is there anything in particular do you want me to try ? I can scan ... |
cc5585f to
62362b2
Compare
|
Hi @Artur-man, Would appreciate it if you tested the windows packages. Also, just an FYI, the effort to provide binary packages just exposed an issue with the configure script. The generated binary for linux is overly large. The CMake code in the configure script was missing some settings that greatly reduce the binary size. You've set us on a path of improvement. |
Always a pleasure XD ok I will bring out the windows machine tonight. |
2f479a8 to
5ece5d7
Compare
|
Trying to find the binaries ... shall make my own or can I get it somewhere, probably from your other repo @zivy ? |
|
ah ok ok found it, some reason could not scroll down in the action page earlier. |
|
For now I am getting this error, I have tried dplyr binary to compare and it works ... but below does not: I will try more tomorrow, sorry about the turkish messages it basically says the file is not found. This |
679f14f to
cb9b2da
Compare
|
Hi @Artur-man, Thanks for testing. Took me a while to figure out the issues. When downloading an artifact it is zipped before download, so an artifact named Bottom line, after you unzip and rename the downloaded file you should be able to do: Latest artifacts are here. |
cb9b2da to
f055bf1
Compare
For all triggering events, the workflow builds SimpleITK using the installer, creates binary packages from the results and uploads the artifacts to GitHub. When the triggering event is a tag push that matches the SITK_TARGET (DESCRIPTION file) a draft release is created and the packages are uploaded to it.
Refer to actions using the hash and not the tag that can be moved. Also limit permissions to read-only.
Stripped debug symbols from the binaries to reduce package size. Hides symbols by default in shared libraries and hide inline C++ functions to reduce binary size and prevent symbol conflicts with other packages.
Use a gh-pages branch for GitHub pages hosting of a CRAN-like repository. When a release is created, there is a workflow that automatically updates the CRAN-like repository using the information from the release assets. Note that the CRAN-like repository does not include the binary packages themselves, it forwards to the URLs of the release assets on GitHub.
3ccb4f5 to
b9ba0af
Compare
|
Hello @Artur-man, @blowekamp, @richardbeare I believe this PR is finally working and will enable us to build and distribute binary packages via GitHub using the standard R As I did not want to corrupt the official repository with my trials and I couldn't test this without merging, I have a development repository under my personal account in which I (1) merged the PR (2) tagged a "release" and pushed the tag to the repo which automatically builds the binaries and creates a draft release (3) changed from draft release to an official release which automatically updates the foyer package and the gh-pages branch of the repository (the usage of a gh-pages branch is so that the updates are automatic, the bot can directly push to the branch which we block on the main branch where we require PRs and review). All this functionality can be found on https://github.com/zivy/sitk_r_installer_dev . Please go to the release page and check that the instructions are clear. Try it out and tell me if you encountered problems. Hopefully this is in a good enough spot that we can merge this long running branch. After merging I will work on adding the missing macos-arm releases and writing a short document describing the release process. |
|
Is there a blocker for Mac ARM binaries? |
|
No real blocker I'm aware of. I got things running and tested on the platforms I have access to (linux, macos-intel and windows) so just wanted to finish this PR and then find out if macos-arm is a trivial addition or I encounter a problem. |
Previous CRAN-like structure was not working with standard R install tools. The File specification in the PACKAGES file is appended to the CRAN-like's URL which works if the actual package is on the website but specifying a URL as the File value results in an invalid URL structure (https://cran_likehttps://file_location). Moving to a foyer package approach supports download from arbitrary location. The user installs the SimpleITK foyer package using standard R tools (mimicing a SimpleITK install). Then calls the library(SimpleITK); install_simpleitk() which downloads and installs SimpleITK itself, overwritting the foyer package. Next call to library(SimpleITK) loads the actual SimpleITK package.
b9ba0af to
01ed642
Compare
|
Thanks @zivy, I will try to try installing this again tomorrow morning, and apologies for the silence on my side ... multiple projects and grants (I have a few that I am leading), are ruining my life :D .... one question though, cant we put the foyer package to r-universe ? You can even make your r-universe setup to target a subfolder of a repo ... which is the foyer subfolder under simpleitk_installer ... let me show you an example. Here in our r-universe page ... we target the I can explain more if needed. |
When a tag that matches the SITK_TARGET is pushed
to the repository, binary R packages for all tested platforms and that specific tag are created and uploaded to a draft release on GitHub.
Binary packages built and uploaded by CircleCI
use the GitHub CLI tool and requiere a GitHub
personal access token (PAT) with full repo control. In CircleCI project settings, under
Environment Variables add a variable:
GITHUB_TOKEN = PAT_value.